Multisensor image registration by feature consensus

1 Summary In this paper, we address the problem of registering two images obtained using diierent sensors, elds of view and/or lighting conditions, where conventional approaches relying on feature correspondence or area correlation are likely to fail. The approach presented in this paper eliminates the need for feature matching, and is robust to variations in sensor characteristics, imaging conditions, and elds of view. It is similar in spirit to methods based on the Generalized Hough Transform (GHT), but it eliminates many of the problems (such as lack of robustness and high computational cost) associated with GHT-style methods. We make two basic assumptions: (1) the characteristics of the scene give rise to detectable features such as points and lines in both images, and at least a part of these features are common to both images and (2) the two images can be at least approximately aligned by global 2-D transformation. For a given problem, we select an appropriate transformation (Euclidean, similarity or aane) based on sensor geometry and other criteria. We rst decompose this transformation into a sequence of elementary stages. At each stage, we select an appropriate image feature class, and estimate the value of one transformation parameter by a feature consensus mechanism in which each feature pair is allowed to select the value of the parameter that is consistent with it. The value of the parameter that is maximally consistent with respect to all the feature pairs is considered to be its best estimate. We introduce the concept of parameter observability to formalize this process. A very useful notion, parameter separability, makes it possible in most cases to completely eliminate the need for feature pairings, and instead work with aggregate properties of features determined from each image separately. The global registration achieved by feature consensus should be suucient for many applications such as those employing registration for performing focus of attention. If a more accurate global registration is needed, as in medical applications, the feature consensus result may be used as an initial condition for more elaborate schemes that use feature correspondence (e.g 10]) or multi-dimensional search (e.g. 14]), which require a good initial guesses for the transformation parameters. Methods like deformable template matching could also be invoked for local reenement of the registration. Abstract This paper presents an approach for registering images obtained using diierent sensors, viewpoints or lighting conditions. This approach does not require feature correspondence or …